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Abstract 

The landscape of medical education is continuously evolving, as are the needs of the learner. The appropriate use of 
research and evaluation is key when assessing the need for change and instituting one’s innovative endeavours. This 
paper demonstrates how research seeks to generate new knowledge, whereas evaluation uses information acquired 
from research to make decisions as to how well a particular practice is working and whether or not changes or 
improvements are required (Polit D & Beck C. 2007). 

Ideally evaluation and research should be synergistic as both research and evaluation seek evidence for developing 
practice (Fain J., 2005). However, in reality this is challenging as there can often be a dichotomy between the two. 
Increasingly, medical educators and researchers are working as part of multidisciplinary teams so that a collective 
picture can be generated using ‘evaluative research’. Not only is this a more efficient way of analysing information, 
but it also encourages the sharing of knowledge across professional borders (Mitchell R et al. 2015). 

It is evident that research and evaluation are of the utmost importance and will continue to be a cornerstone of both 
modem day evidence based medicine and medical education. 
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1. Introduction 

The landscape of medical education is continuously evolving, as are the needs of the learner. During the past decade, 
medical education has undergone significant reform to cater for the needs of the modem day students of ‘Generation 
Y’ (Strauss & Howe, 2000), who are information rich but time poor (Taylor & Sheehan, 2011). In general, there has 
been a shift from the traditional teacher centred approach to more student centred learning, which is characterised by 
active participation and autonomous learning (Hedge et al, 2011). This was bom from the demand for a dynamic 
curriculum that is receptive to global trends and more dependent on technology so that students can take charge of 
their own development (Larson et al, 2009). 

However, one must ask how medical educators became aware of the needs and evolving expectations of students? 
Furthermore, how did they assess the merits and limitations of implementing such changes? This paper discusses 
how an appropriate use of research and evaluation is key when assessing the need for change and instituting one’s 
innovative endeavours. 

In this article, we will firstly explore the world of research in medical education and its origins. Secondly, we will 
analyse the relationship between research and evaluation. Finally, we will briefly examine some of the models of 
evaluation available today. 

2. Research 

When we delve into the world of research, we see that its use in the arena of medical education is relatively new. The 
first fully funded sustained research into improving teaching and learning in medical schools was in the 1950’s, 
nearly half a century after the famous Flexner report (Kuper et al, 2010), (Hitchcock, 2002), (Flexner, 1910). Despite 
the late start, this area has flourished and its progression is reflected in the innumerable medical education and 
research journals available today. 

One can only postulate as to what prompted this explosive growth in medical education research but it’s felt that a 
number of socio-historic factors converged and led to an appreciation of the fundamental obligation for the 
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advancement of knowledge within the medical community (Deitrick & Berson, 1953). Many of these factors are still 
drivers of research today such as; the exponential growth in medical knowledge, increasing importance of scientific 
research, concerns about accountability for and control of medical education (Albert, 2004), (Albert et al, 2007). 

In modern day medicine, research is of the highest priority, not only within academic medicine but in everyday 
clinical practice. The Irish Medical Council (IMC) now stipulate that at least one research audit must carried out 
annually in order to remain on the IMC register (Irish Medical Council, 2015). The importance of research has been 
further emphasised in recent national reports such as the Hunt report, which emphasised the importance of research 
as one of the core competencies of a modem day doctor (Hunt et al, 2011). After all, without research and knowledge 
advancement, we cannot possibly hope to remain at the forefront of best international practice nor can we attract 
enterprise or investment. With this in mind, many universities such as Trinity College Dublin (TCD) now offer an 
intercalated Master’s program to 3 rd year medical students in order to foster a culture of research at an early stage in 
their career (Webster & Kenny, 2011). This ethos continues in the postgraduate phase where many governing 
bodies such as the Royal College of Physicians Ireland (RCPI) and the Royal College of Surgeons in Ireland (RCSI) 
are committed to formally integrating medical research into postgraduate training programs (Royal College of 
Physicians Ireland, 2015). 

Research can take on many forms but ultimately it involves the systematic investigation into sources in order to 
establish facts and reach new conclusions (Oxford Dictionary, 2015). Research has become one of the cornerstones 
of modem day medical practice as we need to be able to make informed decisions based on best evidence rather than 
on anecdotal evidence so that we can offer our patients the optimal treatment course. Broadly speaking, quantitative 
and qualitative research methods are used to address research problems (Figure No. 1). Although both of these 
methods are inherently different in their approach i.e. quantitative research is based on a quantifiable measurement 
process, whereas qualitative research is descriptive and subjective, they can both contribute to the evidence base in a 
complimentary manner (Tavakol & Sandars, 2014). Traditionally, clinical medicine had held quantitative research in 
higher esteem as it is based on statistical figures and so is more tangible than qualitative research. However, in recent 
years, we have seen a growing appreciation of qualitative research, which is used more often in the arena of medical 
education (Tavakol et al, 2008). It is clear that there is no ‘one size fits all’ approach to research methods as each 
approach to inquiry is appropriate in a different research setting and often a mixed methods approach is utilised in a 
bid to maximise new insights into a particular area. 



Quantitative Qualitative 




Figure 1. Quantitative versus Qualitative Approaches 
Adapted from Quantitative vs. Qualitative Measures, available at 
http://baselinesupport.campuslabs.com/hc/en-us/article_attachments/201672385/Quantitative_vs_Qualitative.jpg 
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3. Relationship between Research and Evaluation 

Evaluation is a form of applied research that scrutinises how well a particular practice, program or policy is working 
(Tavakol et al, 2010). Whereas research generally seeks to generate new knowledge, evaluation uses information 
acquired from research to make decisions as to how well a particular practice is operating and whether or not 
changes or improvements are required (Figure No. 2) (Polit & Beck, 2010). 

Ideally evaluation and research should be synergistic as both research and evaluation seek evidence for developing 
practice (Fain, 2005). However, in reality this can be challenging as there can often be a dichotomy between research 
and evaluation. For example, evaluators tend to focus on context, localised results and the interests of the 
stakeholders, whereas researchers focus more on generating or building on existing knowledge, replication of results 
and methodological rigour (Morrison, 2003), (Holzemer W, 1980). However, we are increasingly seeing medical 
educators and researchers working as part of multidisciplinary teams so that a collective picture of a particular 
practice or topic can be generated using ‘evaluative research’ (Polit & Beck, 2010). This is particularly pertinent in 
planning medical education programs and many postgraduate training bodies have now formed taskforces in a bid to 
improve postgraduate training in Ireland so that we can try to retain Irish graduates within the Irish medical system 
(Ferris & Joyce, 2015). These teams usually comprise of medical educators, consultants and trainees from the 
particular training scheme in question and government representatives who are responsible for funding. Not only is 
this a more efficient way of analysing information and enhancing a particular program or practice, but it also 
encourages the sharing of knowledge across professional borders (Mitchell et al, 2015). 



Figure 2. Pictorial Representation of the Relationship Between Research and Evaluation 

Adapted from Michael Quinn Patton’s Evaluation 5 Cartoons. (2014) Available at 
http://freshspectrum.com/wp-content/uploads/2014/07/wpid-Photo-201407150621494.jpg 

4. Evaluation Models 

As we have discussed so far, evaluation is a key component of medical education and medical research as it fosters a 
culture of both accountability and improvement. Over the last decade there has been a growing appreciation of the 
role of evaluation in both public and private decision making processes. This has been reflected in national strategic 
planning documents, which have included evaluation as a key element in not only change management, but in 
meeting the needs and expectations of service users (Figure No.3). 
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Figure 3. Health Service Executive Change Model 

Adapted from the Irish Health Service Executive Policy (2008). A user’s guide to managing change in the Health 

Service Executive, Dublin. Available at 

http://hse.ie/eng/staff/Resources/hrstrategiesreports/Improving_our_Services,_A_Guide_to_Managing_Change_in_t 

he_the_HSE_-_Oct_2008.pdf 

Evaluation in education is well established and dates back to the influential Ralph Tyler almost a century ago (Joyce, 
2010), (Tyler, 1930). Today, there are countless evaluation models available, with each model having inherent pros 
and cons. These models have evolved from various theories underpinning evaluation, namely reductionism, systems 
theory and complexity theory (Yardley & Dornan, 2012). Some of the more well-known approaches to evaluation 
include naturalistic evaluation, fourth generation evaluation, Kirkpatrick’s model, Jacobs’ model, CIPP model, logic 
model and more recently, empowerment and self-evaluation models (McNamara et al, 2010). The addition of more 
modem approaches such a self-evaluation is particularly interesting as it utilises one’s own critical thinking skills, 
which in turn encourages both personal and professional development. In practice, a combination of methods is often 
used in order to enhance the quest for improvement and professional development. 

Let us look briefly at some of the more commonly used evaluations models in medical education. The Kirkpatrick 
evaluation model was introduced over four decades ago and has been used extensively for evaluating educational 
programs (Yardly & Dornan, 2012). Originally, it consisted of four levels (reaction, learning, impact and results) 
with a fifth level being added later by Philips to measure return on investment (Figure No. 4) (Philips, 2003). It is an 
outcome based model and its main advantage is that it provides a clear framework for analysing educational 
programs. However, one has to be cognisant of the fact that it does not take into account factors that can affect 
learning outcomes such as motivation, knowledge base etc (McNamara, 2010). It has also been criticised for 
assuming that there is a linear relationship between educational programs and outcomes, which isn’t always the case 
(Sufflebeam & Shinkfield, 2007). 
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Figure 4. Kirkpatricks Evaluation Model 

Adapted from Salas E. (2009) Patient safety training evaluations; reflections on level 4 and more. Available at 
http://archive.ahrq.gov/news/events/conference/2009/salas/index.html 

The Logic model is a linear evaluation model that can also be used for planning and evaluating educational programs 
(Weiss, 1972). Its uncomplicated design makes it an attractive choice (Figure No. 5). It consists of four basic 
components (Inputs, activities, outputs and outcomes) but it can be expanded by adding feedback loops or adding 
tiers if a more complex analysis is required (Frechtling et al, 2007), (Patton, 2011). 



Figure 5. Logic Model 

Adapted from the Impact Value Chain basic logic model developed by Weiss, C.H. (1972). Available at 
http://4.bp.blogspot.com/-JUxl2CcG2mk/VRnv08XfZRI/AAAAAAAAAIs/w3zN6CQ9Hkg/sl600/BE_diagraml.pn 

g 

The CIPP (context/ input/ process/product) model is useful as it can be adapted to both educational and 
non-educational settings (Figure No. 6) (Stufflebeam, 1983). It is a process based thorough model of evaluation that 
can be applied at any stage of a program development i.e. when planning a new program, assessing a mature 
program or evaluating a program after its completion. Its main advantage is that it is non-liner and so provides a 
dynamic framework for looking at many different aspects of an educational program (Figure No. 6). However, it 
requires careful planning and is labour intensive as multiple data collection methods are often required in order to 
carry out an efficient CIPP evaluation. 
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Figure 6. CIPP Model 

Adapted from Stufflebeam’s CIPP Model 1983. Available at 
http://arcmitO 1 .uncw.edu/jonesi/Images/CIPPEvaluationModel.png 

As with most methods of evaluation, there is no one ‘perfect’ evaluation model, however, it is important to choose a 
model that will help you the collect information needed to determine the merit or worth of your educational program 
(Cook, 2010). 

5. Conclusion 

The landscape of medical education is continuously evolving, as are the needs of the learner. The appropriate use of 
research and evaluation is key when assessing the need for change and instituting one’s innovative endeavours. This 
paper illustrates how research generally seeks to generate new knowledge, whereas evaluation uses information 
acquired from research to make decisions as to how well a particular practice is operating and whether or not 
changes or improvements are required (Polit & Beck, 2007). 

Ideally evaluation and research should be synergistic as both research and evaluation seek evidence for developing 
practice (Fain, 2005). However, in reality this is challenging as there can often be a dichotomy between the two. 
Increasingly, medical educators and researchers are working as part of multidisciplinary teams so that a collective 
picture can be generated using ‘evaluative research’. Not only is this a more efficient way of analysing information, 
but it also encourages the sharing of knowledge across professional borders (Mitchell et al, 2015). 

It is evident that research and evaluation are of the utmost importance and will continue to be a cornerstone of both 
modern day evidence based medicine and medical education. 
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